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ABSTRACT 

This study mapped the correspondence between 
indicators included in "Educational Outcomes and Indicators for 
Students Completing School , M developed by the National Center on 
Educational Outcomes (NCEO) , and indicators included in national data 
collection programs. These data collection programs included such 
efforts as the National Assessment of Educational Progress, the 
National Longitudinal Transition Study of Special Education Students, 
the National Education Longitudinal Study of 1992, the National 
Health Interview Survey, and others. Results indicated that important 
school completion outcomes for all students are substantially 
represented in the national data collection system. Across 13 
national data collection programs, 91 percent of the NCEO school 
completion outcome indicators were found to be represented by at 
least one measure. However, it is currently not possible to produce 
comprehensive reports about the status of students with disabilities 
due to the significant exclusion of such students from data 
collection programs and V a variable or nonexistent identification of 
such students in national data collection programs. Recommendations 
are provided to realize the potential for producing useful 
policy-relevant information regarding school completion outcomes for 
students with disabilities* (Contains 22 references.) (JDD) 
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Abstract 

Reform initiatives throughout the entire educational system are shifting the focus toward 
outcomes and quantifiable data. With increasing frequency, the data needed to monitor and evaluate 
these reforms are being drawn from the nation's existing data collection programs. This 
investigation mapped the correspondence between indicators included in the NCEO comprehensive 
system of Educational Outcomes and Indicators for Students Completing School and indicators 
included in national data collection programs. 

The results indicated that important school completion outcomes for all students are 
substantially represented in the national data collection system. Across 13 national data collection 
programs, 91% of the NCEO school completion outcome indicators were found to be represented 
by at least one measure. However, it is currently not possible to produce comprehensive reports 
about the status of students with disabilities due to the significant exclusion of such students from 
data collection programs and the variable or nonexistent identification of such students in national 
data collection programs. Recommendations are provided to realize the potential present for 
producing useful policy-relevant information regarding school completion outcomes for students 
with disabilities. 
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Matching Information in National Data Collection Programs to a Model of 
School Completion Outcomes and Indicators 

The National Center on Educational Outcomes (NCEO) for students with disabilities was 
established in October, 1990 to work with state departments of education, national policy-making 
groups, and others to facilitate and enrich the development and use of indicators of educational 
outcomes for students with disabilities. It is believed that responsible use of indicators will enable 
students with disabilities to achieve better results from their educational experiences. 

One of the four major strategic goals of the NCEO is to enhance the availability and use of 
outcomes information in decision making at the federal and state level. A variety of activities are 
subsumed under this goal. Two of the activities focus on reviewing the characteristics of major 
data sets in the national education data system as a prelude to secondary analysis of the data sets. 
The purpose of these activities is to determine the feasibility of extracting quality and credible 
policy-relevant information on the educational status and performance of students with disabilities 
(McGrew, Spiegel, Thurlow, Ysseldyke, Bruininks, & Shriner., 1992). The results of this critical 
review process will contribute to the NCEO strategic goals by: 

1. Determining the extent to which the current national data collection programs consider 

students with disabilities when planning and implementing data collection. 

2. Reviewing the extent to which students with disabilities are included or excluded in 
national data collection programs as a result of sample design and inclusion/exclusion 
procedures. 

3. Reviewing the inclusion and exclusion procedures used in national data collection 
programs. 

4. Determining the depth and breadth of outcomes included in national data collection 
programs and the extent to which these outcomes approximate a conceptual model for a 
comprehensive system of indicators. 

5. Reviewing the means by which national data collection programs describe the functional 
characteristics of students with disabilities. 

Secondary data analysis will be completed using existing data. It is hoped that the results 
of the review process indicate that sufficient quality data exist in the nation's data collection system 
to support credible analyses. The completion of these analyses will contribute to other NCEO 
strategic goals by providing information to include in the NCEO synthesis reports on the 
educational outcomes of children and youth with disabilities. 

NCEO findings related to the first three gocJs listed above have been reported (McGrew, 
Algozzine, Spiegel, Thurlow, & Ysseldyke, 1993; McGrew, Thurlow, Shriner, & Spiegel, 1992; 
McGrew, Thurlow, & Spiegel, 1993). This report focuses on NCEO activities related to goal 
number four, the extent to which the indicators included in the NCEO comprehensive system of 
Educational Outcomes and Indicators for Students Completing School (Ysseldyke, Thurlow & 
Gilman, 1993) are included in national data collection programs. The purpose of this activity is to: 
(a) highlight potential gaps in the current national education data collection system, a system that 
has not had the benefit of evolving from an a priori conceptual model, and (b) provide an 
organizational framework from which to conduct secondary analysis (McGrew, Spiegel, Thurlow, 
Ysseldyke, Bruininks, & Shriner, 1992). The identification of information gaps in the current data 
collection system may serve to stimulate the modification of current data collection programs. It 
also may stimulate the development of new data collection programs that provide for more 
comprehensive conceptually-based national data. Hopefully, such data will provide useful, 
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policy-relevant information for all children. 

The Current Context: Measurement-Driven Education Refom: 

Our nation is becoming "increasingly dependent on statistics for policy analysis and decision 
making" (Andrew, 1984, p. 51). Furthermore, "school reform has riveted national attention on the 
numbers" (Hanford & White, 1991). Reform initiatives throughout the entire educational system 
are shifting the focus toward outcomes and quantifiable data. With increasing frequency, the 
data needed to monitor and evaluate these reforms are being drawn from data bases in the nation's 
existing education data collection system. 

The current national goals and educational indicators movements have produced a flurry of 
activity to identify data bases that include indicators to help monitor progress toward goal 
attainment. The Special Study Panel on Educational Indicators (1991) reported that the success of 
educational reform depends on the development of a "comprehensive education indicators 
information system capable of monitoring the health of the enterprise, identifying problems, and 
illuminating the road ahead" (p.6). Similar calls fcr the improvement of the existing national 
education data system and the development of new components to include in the system have been 
echoed in reports by the National Education Goals Panel (1991a) and the National Education 
Statistics Agenda Committee of the National Forum on Education Statistics (1990). Each of these 
groups has turned toward national data collection programs for indicators to monitor progress 
during the current wave of reform. 

For example, in its report Measuring Progress Toward the National Edu cation Goals: 
Potential Indicators and Measurement Strategies , the National Education Goals Panel (1991a) 
reports on how to measure progress toward the six national education goals by recommending the 
use of indicators from such data sets as the National Education Longitudinal Study (NELS) and the 
National Assessment of Educational Progress (NAEP). The National Education Statistics Agenda 
Committee also recommended using indicators from NELS and NAEP, as well as other data sets, 
to improve the data provided by the national education data system. Other groups looking for 
indicators from existing national data bases as a means to measure progress of children and youth 
include the Council of Chief State School Officers, Joining Forces (a coalition of the American 
Public Welfare Association and the Council of Chief State School Officers), Kids Count , and the 
NCEO, to name but a few. 

In addition to the general education reform movement, recent reform initiatives in special 
education (Skrtic, 1991) are producing increased interest in analysis of existing national data bases. 
Since the passage of PL 94-142 in 1975, there has been over a decade of evaluation studies that 
have focused primarily on the issue of educational access for students with disabilities and 
implementation of the processes embodied in the law. Increasingly the question of " where's the 
beef?" has been asked from both within and outside of special education. Focus has recently 
turned toward evaluating the outcomes of special education, or, "where's the data?" on 
effectiveness (DeStefano & Wagner, 1991). 

The NCEO Conceptual Model of Outcomes And Indicators 

Secondary analyses of large extant data bases "have enormous potential for policy analysis 
and evaluation research" (Meyers & Rockwell, 1984, p. 5) and such analyses are on the increase. 
Analysis of indicators from national education data sets have produced policy-relevant reports such 
as NCES's The Conditio n of Education, OERFs Youth Indicators . Hispanic Education: A 
Statistical Portrait 1990 (De La Rosa & Maw, 1990), the 1990 Chartbo ok: Services for People 
with Disabilities (Amado, Lakin, & Menke, 1990), and the annual National Edu cation Goals 
Report (National Education Goals Panel, 1991b, 1992, 1993). These and other similar reports 
have as their major thrust the communication of information to critical decision makers in the policy 
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arena. Such policy-relevant reports are only possible through the secondary analysis of a number 
of data sets. 

An eight step process is used to guide NCEO secondary analysis activities (McGrew, 
Spiegel, Thurlow, Ysseldyke, Bmininks, & Shriner, 1992). The second step in this process is the 
development or use of a conceptual framework to guide the analyses. A conceptual framework 
serves to guide the analyses and insure that the informational needs and research questions are 
answered (DeStefano & Wagner, 1991). 

The NCEO Conceptual Model of Outcomes (Ysseldyke et al,, 1993) is serving this puipose 
for the NCEO secondary analysis activities. The NCEO model was designed to reflect outcomes 
that apply to all students, not just students with disabilities. Hundreds of educators, policymakers, 
and parents participated in a structured consensus building process (Vanderwood, Ysseldyke, & 
Thurlow, !993) that resulted in the current model (Gilman, Thurlow & Ysseldyke, 1992; 
Ysseldyke et al., 1993). The NCEO Conceptual Model of Outcomes for School Completion is 
presented in Figure 1. 

The conceptual framework depicted in Figure 1 shows the complete educational model, with 
Educational Resources (Inputs and Contexts) influencing Educational Opportunity and Process. 
These in turn, influence Outcome Domains, which have a return influence on both the resources 
and opportunity/process. The inclusion of two of the outcome domains (i.e., Presence and 
Participation; Accommodation and Adaptation) in the model has been controversial. A detailed 
discussion of issues surrounding the development and ongoing refinement of the NCEO model is 
presented in Ysseldyke et al. (1993). 

The conceptual model is extended by the identification of outcomes within each outcome 
domain and indicators of the respective outcomes. Outcomes are the results of interactions 
between the students and the education system. Indicators are numbers or other symbolic 
representations that can be used to determine whether desired outcomes are achieved. Figures 2 
through 9 present the outcomes and outcome indicators for each outcome domain for the NCEO 
model developed for students who have completed school (Y sseldyke et al., 1993). 

The outcomes and indicators presented in Figures 2 through 9 served as the basis for the 
analyses summarized in this report. This investigation mapped the correspondence between the 
indicators of outcomes in the NCEO school completion model and indicators contained in recurring 
national data collection programs. 

Method 

Selection of Data Collection Programs 

Thirteen national data collection programs were analyzed in the current investigation. These 
data collection programs represent a subset of 28 national data collection programs identified by the 
NCEO as being potentially useful in the extraction of policy-relevant information on the educational 
status and performance of students with disabilities (McGrew, Thurlow, Shriner, & Spiegel, 
1992). The complete set of 28 data collection programs was targeted based on their: (a) potential 
usefulness in providing indicators of outcome domains in the NCEO conceptual model of 
educational outcomes for children and youth with disabilities (Ysseldyke et al., 1993), and (b) 
prominence in current efforts to monitor progress toward the attainment of national education 
goals. 

The 13 data collection programs included in the current investigation are briefly described in 
Table 1. Since the purpose of this investigation was to determine the extent to which the NCEO 
school completion outcomes and indicators are included in recurring national data collection 
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programs, only those data collection programs that currently occur, or are planned to occur, on a 
regular basis were selected. In addition, only those data collection programs that included 
individuals of school completion age were included. 

For example, the National Assessment of Educational Progress (NAEP), Current Population 
Survey (CPS), and National Health Interview Survey (NHIS) are data collection programs that 
occur on a regular basis and include individuals of this age group in their samples. These three 
data collection programs were included in this study. This contrasts with other data collection 
programs that may contain important outcomes and indicators but which are not recurring 
programs. For example, although the National Adolescent Student Health Survey (NASHS) 
contains numerous outcomes and indicators relevant to portions of the NCEO Conceptual Model of 
Outcomes, this data collection program was completed in 1988 and is not designed to be an 
ongoing, routine data collection program. 

There were two exceptions to the above selection criteria. The National Education 
Longitudinal Study of 1988 (NELS:88) was selected since it represents the third in a series of 
longitudinal studies established as part of the National Center on Education Statistics (NCES) 
National Longitudinal Studies program (Nelson, 1991). Although the NELS:88 study will 
eventually end, a similar data collection program will most likely take its place as part of NCES f s 
series of educational longitudinal studies. The National Longitudinal Transition Study (NLTS) of 
Special Education Students was the second exception. Although a fixed duration longitudinal data 
collection program, NLTS represents the first large-scale national suivey of students with 
disabilities. It is important to determine the extent to which similar specialized national data 
collection programs of the future, or, other data collection programs that might incorporate some of 
NLTS f s variables, results in broader coverage of the NCEO outcomes and indicators. 

The relevant methodological and technical reports for each data collection program were 
obtained from the sponsoring agency. The section of each data collection program's codebook or 
user manual that presented the assessment instruments served as the source documents for the 
subsequent analyses. 

Analysis of Data Collection Programs 

The individual items in each data collection program's assessment instruments were 
compared to each NCEO outcome and outcome indicator listed in Figures 2 through 9. The goal 
was to identify assessment items that corresponded closely to the NCEO outcome indicators. If 
one or more survey items in a data collection instrument were judged to be a close match to an 
NCEO outcome indicator, the data collection program was classified as providing potential 
indicators for that outcome in the NCEO Conceptual Model. Two individuals completed this 
review process for each data collection program. 

As is often the case, the variables and indicators of concepts included in data collection 
programs typically do not provide an exact match to the indicators needed for secondary analysis 
(McGrew et al., 1992). Given that secondary analysts must often use less than perfect "surrogate" 
measures that were originally developed for different purposes, conceptually similar or 
approximate indicators were considered to be a "match" with an NCEO outcome, when so judged 
by group consensus. Discrepancies between the reviewers were resolved by group consensus, 
and/or by review by a third individual. The following examples provide an illustration of the 
reasoning used in the model-indicator mapping process. 

The NCEO Physical Health domain (Figure 4) includes the outcome of "Makes Healthy 
Lifestyle Choices." One of the indicators of this outcome is "Percent of students who indicate that 
they use tobacco products." The Youth Risk Behavior Surveillance Survey (YRBS) includes three 
items that closely match the intent of this NCEO outcome indicator. The three YRBS survey 
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Figure 1 
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Figure 2 

d = OUTCOME 

/(^DOMAIN 

Presence and Participation 



A iL' * INDICATOR 



f Ai Is present In school 



Absenteeism rate during last year of school 
(differentiated for reasons of suspension, 
medical/health, truancy, and other) 



Percent of students excluded from their typical school 



placement 



/;. Percent of students attending residential settings (in and 
/ c out of state), separate schools, separate classes 



«^A2 Participates t w / Percent of time students participate actively in general 

V, & : ... „ ■ education classrooms during last year of school 



r— /b 



Percent of time students participate actively in community 
activities during la* i year of school 

Percent of time students participate actively in 
ii-JL . extracurricular activities during last year of school 

f 

I 
i 
r 

/f Percent of students who have had employment experience 



Percent of students who participate in district, state, and 
national testing programs 



A* ... (through school programs) before leaving school 



A3 Completes school >^^»»r.7^ /* Percent of students who graduate with a diploma 

I 

: Percent of students who earn a certificate of 

! Z* ... completion/attendance 

. -c Percent of students who earn GED diploma 

»«.• »•■ d Percent of students who drop out 
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Figure 3 

^ rtl ^ MC 4 a INDICATOR 

s OUTCOME /-b 

Accommodation and Adaptation 



Makes adaptations, 
accommodations, or 
compensations 
necessary to achieve 
outcomes in each of the 
major domains. 



Percent of students who demonstrate adaptation/ 
accommodation/compensation skills required to move 
about in their environments 

Percent of students who demonstrate adaptation/ 
accommodation/compensation skills required to 
communicate 

Percent of students who demonstrate adaptation/ 
accommodation/compensation skills required to read 

Percent of students who demonstrate adaptation/ 
accommodation/compensation skills required to participate 
in activities in home, school, and community environments 

Percent of students who demonstrate adaptation/ 
accommodation/compensation skills required to manage 
personal needs in home, school, and community 
environments 



Demonstrates family - ■ 

support and coping 

skills 

b 



Percent of families prepared to cope with student's 
needs after student leaves school 

Percent of families knowledgeable about community 
resources and programs needed by student 



Percent of families participating in the education of their 
children 
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Figure 4 
ir| = OUTCOME 



DOMAIN ' 



^ r INDICATOR 



Physical Health 



^ 5T : M 0 ^ 5 healthy lifestyle 
choices 



Percent of students who indicate *t they use tobacco 
products 



■chaser .- jj Percent of students who make good nutritional choices 



Percent of students who have abused alcohol or drugs 



in the past year 



Percent of students who indicate they have had 
unprotected sex in the past year 



Percent of students who elect to participate in sports, 
recreational, and/or exercise activities 



. q 2 k aware of basic safety, *u- 
•i. ■' fitness, and health care 
needs 



Percent of students who are aware of basic safety 
precautions and procedures 

Percent of students who are aware of basic fitness 
needs 



Percent of students who are aware of basic health 
care needs 



Percent of students who know when, where, and 
how to access health care 



C3 Is physically fit - m Percent of students who are physically fit 
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Figure 5 

= OUTCOME ^£-> .4. = INDiCATOR 

^DOMAIN 

Responsibility and independence 



r. cats about In the - mr-r— - ' Percent of students who can get to and from a variety 

^£ •nvironmmt ~~ of destinations 

Percent of students who know how to access 
■■ sarss& .-. b community services (e.g„ rehabilitation, counseling, 
~ employment, health, etc.) 

Percent of students who complete transactions 
c (shopping, banking, dry cleaning, etc.) in the 
community 



d 



Percent of students with a d river's license 



Percent of students who can prioritize and set goals and 
persevere toward them 

b ■ Percent of students who manage personal care and safety 

Percent of students who effectively advocate for 
. themselves 

* 

d Percent of students who are likely to engage in lifeiong 
learning 



02 is responsible for self 
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Figure 6 



= OUTCOME 



^DOMAIN 



INDICATOR 



Contribution and Citizenship 



-'.irvl Com P ,,es wlth school 
H^v/ and community rules 



Percent of students who have been suspended or 
subjected to other disciplinary actions 



Percent of students who have been repeatedly 
suspended or subjected to disciplinary actions 



C E2 Knows the significance of 
^ voting and procedures 
necessary to register and 
vote 



Vandalism rate and magnitude 



. ~d Crime rate and magnitude 



i Percent of students who know the significance of voting 



Percent of students who know the procedures necessary 
to register and vote 



E3 Volunteers 



a Percent of students who volunteer time to school, civic, 
community, or nonprofit activities 



9 

ERLC 



10 



15 



Technical Report 7 



Figure 7 



= OUTCOME 



tOOMAIN 



INDICATOR 



Academic and Functional Literacy 



;$F1 Demonstrates competence in 
'K^by' communication 



Percent of students who use and comprehend language 
that effectively accomplishes the purpose of the 
communication 



PF2 



Demonstrates competence in 
problem-solving strategies 
and critical thinking skills 



Percent of students who demonstrate problem-solving and 
critical thinking skills 



F3 



Demonstrates competence 



^ in math, reading and writing 
skills 



Percent of students who demonstrate competence in math 
necessary to function in their current home, school, work, 
and community environments 



Percent of students who demonstrate competence in math 
necessary to function in their next environment 

Percent of students who demonstrate competence in 
reading necessary to function in their current home, 
school, work, and community environments 

Percent of students who demonstrate competence in 
reading necessary to function in their next environment 

Percent of students who demonstrate competence in 
writing necessary to function in their current home, school, 
work, and community environments 

Percent of students who demonstrate competence in 
writing necessary to function in their next environment 



P 4 Demonstrates competence 
- in other academic and 
nonacademlc skills 



Percent of students who demonstrate competence in other 
academic and nonacademic skills necessary to function in 
their current home, school, work, and community 
environments 



Percent of students who demonstrate competence in other 
academic and nonacademic skills necessary to function in 
their next environment 



£.p5 . Demonstrates ~ : 

* competence In 
using technology 



Percent of students who currently apply technology to 
enhance functioning in home, school, work, and 
community environments 

Percent of students who demonstrate competence in using 
technology to function in their next environment 
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Figure 8 



£?2 m OUTCOME 



^JDQUAW 



s INDICATOR 



Personal and Social Adjustment 



Copes effectively with 
£**T7 P*w*>nal challenges, 
frustrations, and 
stressors 



Percent of students who cope effectively with 
personal challenges, frustrations, and stressors 

Percent of students whose behavior reflects an 
acceptance of the consequences for behavior 
(e.g., makes restitution) 



tG2 Has a good self image 



Percent of students who perceive themselves 
as worthwhile 



Percent of students who perceive themselves 
as competent 



Respects cultural 
and individual 
differences 



Percent of students whose behavior demonstrates 
acceptance of cultural and individual differences 



ru G** 8 along with other 
people 



Percent of students who have friends and are a part of a 
social network 



Percent of students who demonstrate skill in interacting 
and in making decisions in social situations, including 
during interpersonal conflict 

Percent of students who engage in productive group work 
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Figure 9 



= OUTCOME 




A 



= INDICATOR 



Student satisfaction with 
high school experience 



<s^es»* 

A 

A 



Percent of students who are satisfied with level of 
achievement 

Percent of students who arc satisfied with what was 
provided in school 

Percent of students who are satisfied with high school 
experience 

Percent of students who are satisfied with progress 
toward achieving educational outcomes 



Parent/guardian ™ 
satisfaction with the 
education that 
students received 



-A 

- A 

- A 

-A 
-A 



Percent of parents/guardians who are satisfied with 
level of achievement 

Percent of parents/guardians who are satisfied with 
what was provided in school 

Percent of parents/guardians who are satisfied with high 
school experience 

Percent of parents/guardians who are satisfied with 
progress toward achieving educational outcomes 

Percent of parents/guardians who are satisfied with the 
extent to which student is prepared to live in society 



Community — 
satisfaction with the 
education that 
students received 



—A 



Percent of community (teachers, policymakers, 
employers, general public) satisfied with students' level 
of achievement 



Percent of community (teachers, policymakers, 
employers, general public) satisfied with what was 
provided in school 



Percent of community (teachers, policymakers, 
c% employers, general public) satisfied with students 1 
progress toward achieving educational outcomes 
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Table 1 

Descriptions of National Data Collection Programs Inclu ! Investigation 



National Longitudinal Transition Study of Special Education Students (Department of Education) - NLTS.87 

A nationally representative longitudinal study of special education students who where in grades 7-12 during 
the 1987 base year sample. By collecting a wide array of information from parents/guardians, school records, 
and school administrators, this data collection program provides descriptive information regarding the 
transition of youth with disabilities from secondary school to early adulthood, and seeks to identify factors 
that contribute to effective transition of youth with disabilities. The first follow-up was completed in 1990. 

National Assessment of Educational Progress 

(Department of Education) - NAEP:88 

NAEP is a nationally representative cross-sectional study designed to monitor the knowledge, skills, 
understanding, and attitudes of the nation's children and youth. This data collection program began in 1969 
and currently assesses different curriculum areas (e.g., reading, writing, mathematics, science, citizenship, 
U. S. History, geography, social studies, art, music, literature, career and occupational development) in 
grades 4, 8, and 12 every two years. 

National Education Longitudinal Study of 1992 
(Department of Education) - NELS:92 

A nationally representative longitudinal study designed to assess the baseline experiences of 8th grade students 

and to relate these experiences to current academic achievement and to later achievement in school and life. 

The base year data collection program started in 1988 and the first and second followups were in 1990 and 

1992. 

National Adult Literacy Survey 

(Department of Education) - NALS:92 

A nationally representative cross-sectional study designed to collect information on the types and levels of 
literacy skills of adults a^vi now uiese skills are distributed across major subgroups. This study assessed the 
prose, document, and quantitative literacy of young adults in 1992. 

National Household Education Survey 

(Department of Education) - NHES:91 

A nationally representative cross-sectional sample of households were surveyed in 1991 to provide national 
data regarding early and adult education issues. The 1991 base year survey targets information on the care and 
education of 3-to8 year old children, and the participation of adults in education activities. 

1900 High School Transcript Study 

(Department of Education) - HSTS:90 

A nationally representative cross-sectional sample of 12th grade students selected from the 1990 NAEP 
survey. The study provided information on the course-taking patterns of 12th grade students. 

National Health Interview Survey 

(Department of Health and Human Services) - NHIS:89 

A nationally representative cross-sectional study designed to provide information on the health of the civilian 
noninstitutionalized U.S. population (birth through adulthood). This survey has been completed annually 
since 1957. While the same basic demographic and health-related information is collected each year, 
additional information on special health topics (e.g., AIDs, aging, etc.) may be covered in any one survey. 

National Survey of Family Growth 

(Department of Health and Human Services) - NSFG:88 

A nationally representative cross-sectional sample drawn from households involved in the National Health 
Interview Survey (NHIS). The 1988 cycle included women from 15-44 years of age who where included in 
the 1986 NHIS. The study provides national data on the demographic and social factors associated with 
childbearing, contraception, adoptions, and maternal and child health. 
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Youth Risk Behavior Survey 

(Department of Health and Human Services) - YRBS:93 

A nationally representative cross-sectional sample of students in grades 9-12. As part of the Youth Risk 
Behavior Surveillance System (YRBSS), this study is designed to periodically (every two years) measure the 
prevalence of priority health-risk behaviors among the nation's youth, and to assess whether these behaviors 
change over time. 

National Crime Survey 

(Department of Justice) - NCS:86-89 

A nationally representative cross-sectional sample (collected on a three year cycle) of household members 
from age 12 and above. The study is designed to collect data on personal and household crime victimization. 

Current Population Survey. March Suppleme nt 

(Department of Commerce) - CPS 

A nationally representative cross-sectional study designed to collect information on the employment situation 
and demographic status of the complete U. S. population (birth through adulthood). The March Supplement 
is specifically designed to gather data on work experience, income, noncash benefits, and population 
migration. Data collection in this program has been occurring annually since the 1940's. 

National Household Survey on Drue Abu se 

(Department of Health and Human Services) - NHSDA:93 

A nationally representative cross-sectional study designed to collect information on the use of tobacco, 
alcohol, illicit drugs, and nonmedical use of prescription drugs. Data collection in this program has been 
occurring on a biennial basis since 1971. 

Monitoring the Future 

(Department of Health and Human Services) - MF:93 

A nationally representative longitudinal study designed to collect information on drug use and attitudes about 
drugs, views about personal lifestyles, confidence in social institutions, intergroup and interpersonal attitudes, 
concerns about conservation and ecology, and other social and ethical issues. High school seniors have been 
the base year sample for this annual survey since 1975. Followup surveys are conducted annually for each 
class for up to fourteen years. 
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questions are: "Durin , the past 30 days, on how many days did you smoke cigarettes?"; "During 
the past 30 days, did you use chewing tobacco, such as Redman, Levi Garrett, or Beechnut, or 
snuff, such as Skoal, Skoal Bandits, or Copenhagen?"; and "During the past 30 days, on the days 
you smoked, how many cigarettes did you smoke per day?" Few would argue that these three 
survey questions could be used as items to estimate the percent of students who use tobacco 
products. 

An example of a more difficult mapping judgment can also be found for YRBS in the NCEO 
outcome domain of Physical Health. As listed in Figure 4, the NCEO Physical Health outcome 
domain includes the outcome "Is Physically Fit," with the indicator "Percent of students who are 
physically fit." YRBS includes a series of questions that inquire about a student's physical 
activity, including self-report of how often the individual exercises, walks or bicycles for at least 
30 minutes at a time, and frequency and time involvement in physical education classes. Although 
these and other YRBS items focus on participation in physical activities, activities that can 
contribute to good physical fitness, one cannot infer or guarantee "fitness" from participation 
alone. Thus, YRBS was judged not to include reasonably good indicators that approximate the 
NCEO outcome of "Is physically fit." When difficult model-indictor mapping decisions were 
necessary, the final criterion used to make judgments was whether the NCEO staff would feel 
comfortable in using the variables as proxies for the NCEO outcome indicators in secondary data 
analysis. 

Method 

The results of the mapping process are presented by NCEO School Completion outcome 
domains in Tables 2 through 9. 

In the NCEO domain of Presence and Participation, each of the 12 NCEO indicators is 
present in at least one of the data collection programs reviewed (see Table 2). Most indicators were 
available in several data bases, from both educational and noneducational data collection programs. 
Given that the areas of school attendance and completion have been the focus of much previous 
research, it is not surprising that four Presence and Participation indicators were measured by four 
data collection programs. These indicators were "absenteeism rate" (Ala), "percent of students 
graduating with a diploma" (A3a), "percent of students earning a GED" (A3c), and "percent of 
students who drop out" ( A3d). The least represented indicators were "percent of time students 
participate actively in general education classrooms during last school year" (A2a) and "percent of 
students who participate in district, state, and national testing programs" (A2d). An indicator 
specifically focused on students in special education (i.e, A2a; percent of time students participate 
actively in general education classrooms) was available only in NLTS:87. 

Only two of the data collection programs had items that matched any of the NCEO 
Accomodation and Adaptation indicators (see Table 3). While all eight in * ;rs were available in 
this domain, NLTS:87 exclusively provided most of these measures. In th^ outcome area of 
"makes adaptations, accommodations, or compensations necessary to achieve outcomes in each of 
the major domains" (B 1), only NLTS:87 contained any indicators. The outcome areas regarding 
family support and coping skills (B2) only had indicators available in NELS:92 and NLTS:87. 

Nine of the ten NCEO Physical Health indicators (90%) had measures available (see Table 
4). Two indicators each had measures in five different data collection programs. These were 
• "percent of students who indicate they use tobacco products" (CI a) and "percent of students who 
have abused alcohol or drugs in the past year" (Clc). Multiple coverage of these "negative" 
indicators is contrasted with less coverage of "positive" indicators. The indicator of "percent of 
students who know when, where, and how to access health care" (C2d) was not addressed in any 
of the data bases. Most of the data collection programs that included indicators were sponsored by 
the Department of Health and Human Services (NHIS:89, YRBS:93, NHSDA:93, MF:93), with 
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YRBS:93 and MF:93 presenting the most extensive coverage. 

Seven of eight indicators (88%) in the domain of Responsibility and Independence were present in 
the reviewed data bases (see Table 5). No measure of "percent of students who effectively 
advocate for themselves" (D2c) was found in the data collection programs. Most indicators were 
covered by only one or two data bases, and no single data collection program covered more than 
50% of the indicators. Indicators for "percent of students who know how to access community 
services" (Dlb) were only available for females in NSFG:88. 

In the domain of Contribution and Citizenship, all seven indicators had measures available 
(see Table 6). In the outcome areas of "compliance with school and community rules" (El) and 
"volunteerism" (E3), indicators were available in multiple data bases. In particular, "crime rate and 
magnitude" (Eld) and "percent of students who volunteer time to school, community, or nonprofit 
activities" (E3a) had measures in four to five data bases. The outcome domain of "knows 
significance of voting and procedures necessary to register and vote" (E2) was least represented in 
the data collection programs. The NELS:92 and MF:93 data collection programs provided 
measures on most indicators in the Contribution and Citizenship domain. 

As would be expected, in the domain of Academic and Functional Literacy the data collection 
programs of the Department of Education (NLTS:87, NAEP, NELS:92, NALS:92, NHES:91; 
HSTS:90) provided measures for all indicators, while noneducational programs provided no 
measures (see Table 7). Indicators were available in multiple data collection programs in the 
traditional school subjects of reading, writing, and mathematics (F3). The outcome areas of 
competence in communication (Fl), problem solving and critical thinking skills (F2), and academic 
and nonacademic skills (F4) had less coverage. In the outcome area of competence in using 
technology (F5), only NLTS:87 included indicators. 

In the domain of Personal and Social Adjustment, seven of the eight indicators (88%) had 
measures available (see Table 8). No measure for "percent of students whose behavior reflects an 
acceptance of the consequences for behavior (e.g., makes restitution)" (Gib) was present in the 
data collection programs. Most indicators had measures in only one or two data bases, with most 
coverage coming from NELS:92 and MF:93. 

Satisfaction (see Table 9) had the least coverage of all the NCEO domains. Eight of twelve 
(67%) NCEO indicators were found across data collection programs. Only two data bases, 
NELS:92 and MF:93, had measures in this domain. No measures were available for indicators of 
satisfaction with students level of achievement (HI a, H2s, H3a) or "percent of community 
(teachers, policy makers, employers, general public) satisfied with students 1 progress toward 
achieving educational outcomes" (H3c). 

Table 10 presents a summary of the availability of NCEO School Completion outcome 
indicators across data collection programs. Overall, the coverage is high, with 91% of the NCEO 
indicators having a measure in at least one of the 13 data collection programs reviewed. When the 
NLTS data collection program (the only program specifically focused on individuals with 
disabilities) is excluded from the table, this percentage drops to 81%. Most of this difference 
occurs in the domain of Accommodation and Adaptation; when NLTS is included, 100% of the 
indicators in this domain are present, when NLTS is excluded, only 38%. By itself, NLTS covers 
48% of all 77 NCEO indicators. This figure is surpassed only by NELS:92, which covers 53% of 
all NCEO indicators. While the overall percentage of indicators present is high, this figure 
represents the combination of 13 major national data collection programs. Over one-third of all the 
indicators had just one measure across all data bases; 33% had two or three measures, and 22% 
had four or more. Nine percent had no measures available. 

The data base/indicator ratios presented in Table 10 indicate that the breadth of coverage 
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Table 10. Availability of NCEO School Completion Level Outcome Indicators in Select National Data Collection 
Programs 



= — 




Number of indicators 


Percent of indicators 


Database | 


NCEO Outcome Domains 


Number of 


for which data may be 


for which data may be 


to indicator | 




indicators 


available in reviewed 


available in reviewed 


ratio^ 1 




in domain 


data bases 1 


data bases 1 




A. Presence and Participation 


12 


12 


100% 


3.9 


B. Accommodation and Adaptation 


8 


8 


100% 


1.1 I 


C. Physical Health 


10 


9 


90% 


2.5 


D. Responsibility and Independence 


8 


7 


88% 


1.8 


E. Contribution and Citizenship 


7 


7 


100% 


2.8 


F. Academic and Functional Literacy 


12 


12 


100% 


3.4 


G. Personal and Social Adjustment 


8 


7 


88% 


1.6 


H. Satisfaction 


12 


8 


67% 


0.8 K 


All NCEO Outcome Domains 


77 


70 


91% 


2.6 | 



data bases that were reviewed are the National Longitudinal Transition Study of Special Education Students, 

1987 (NLTS); National Assessment of Educational Progress (NAEP); National Education Longitudinal Study of 

1988 (NELS:88); National Adult Literacy Survey, 1992 (NALS); National Household Education Survey, 1991 
(adult) (NHES); High School Transcript Study, 1990 (HSTS); National Survey of Family Growth (NSFG); Youth 
Risk Behavior Survey, 1990-91 (YRBS); National Crime Survey, 1986-89 (NCS); and Current Population Survey 
(CPS); National Household Survey of Drug Abuse, 1993 (NHS DA); Monitoring the Future, 1993 (MF). 

^This number represents the average number of data collection programs with measures for each indicator in an 
outcome domain. Thus, for example, there are an average of 3.9 data collection programs with measures of 
indicators in the Presence and Participation domain. 
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(average number of data collection programs p-. indicator) is the greatest in the NCEO outcome 
domains of Presence and Participation (3.9), Academic and Functional Literacy (3.4), Contribution 
and Citizenship (2.8), and Physical Health (2.5). The outcome domains of Responsibility and 
Independence (1.8), Personal and Social Adjustment (1.6), Accommodation and Adaptation (LI), 
and Satisfaction (0.8) are not as well represented. 

Discussion 

Indicators of important school completion outcomes for all students are substantially 
represented in our national data collection system. Across 1 3 national data collection programs 
that include individuals at the completion of their secondary schooling, 91% (70 of 77) of the 
NCEO outcome indicators are represented by at least one measure. When the non-recurring 
National Longitudinal Transition Study of Special Education Students (NLTS) is excluded, the 
coverage is approximately 10% less (81% of the NCEO outcome indicators are represented). 
Approximately half of all the NCEO indicators are present in two or more of the 13 data collection 
programs. With approximately 80% of the NCEO school completion outcome indicators present in 
our recurring national data collection system, the potential exis's for producing comprehensive and 
conceptually organized policy reports regarding the status of all students as they complete school. 

As would be expected given the different purposes of data collection programs, the 
availability of measures that approximate the NCEO indicators varies widely. The NLTS:87 and 
NELS:92 data collection programs are the most promising data sets for providing information 
regarding the NCEO outcome domains. These two data collection programs each include measures 
that are conceptually similar to approximately half of the NCEO school completion outcome 
indicators. However, analysts who seek to provide information regarding the complete NCEO 
model will need to draw indicators from most all of the 13 data collection programs reviewed in 
this study. 

The conceptual and technical problems typically encountered when comparing and/or 
merging information from different data collection programs (McGrew, Spiegel, Thurlow, 
Ysseldyke, Bruininks, Deno, & Shriner, 1991) will most likely make the final percent of usable 
NCEO indicators less than the 80% to 90% figures reported in this investigation. Approximately 
one-third of all the NCEO school completion indicators are only present in one data collection 
program. Given the certainty that some of these single indicators will be unusable due to the 
technical and methodological problems encountered in secondary data analysis (McGrew et al., 
1991), the final percent of NCEO indicators available for analysis will be under 80%. 

The data baseAndicator ratios for the different NCEO outcome domains suggest that the 
potential problem of unusable indicators may be most significant in the areas of Satisfaction (0.8) 
and Accommodation and Adaptation (1.1), and to a lessor extent Personal and Social Adjustment 
(1.6) and Responsibility and Independence (1.8). Outcome domains that are likely to still be 
represented by usable indicators after indicator attrition due to technical problems are Presence and 
Participation (3.9), Academic and Functional Literacy (3.4), Contribution and Citizenship (2.8), 
and Physical Health (2.5). 

Even if the problems in comparing and combining information from different data bases can 
be resolved, the ability to extract useful information regarding students with disabilities may be all 
but impossible. Students with disabilities are often missing from national data collection 
programs, and if present, they cannot be consistently identified for analysis. An investigation of 
the exclusion of students with disabilities in national data collection programs found that in 
educational data sets, 40 % to 50 % of the population of students with disabilities is typically 
excluded (McGrew, Thurlow, & Spiegel, 1993). In noneducational data collection programs, 
students with disabilities are included to a greater degree. However, even when present in the 
samples, individuals vith disabilities often cannot be identified with any consistency due to the 
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lack of disability-specific identifying variables or the use of different disability categories across 
data collection programs (McGrew, Algozzine, Spiegel, Thurlow, & Ysseldyke, 1993). 

In conclusion, the potential exists for producing policy-relevant information on the school 
completion outcomes of students with disabilities. A significant number (over three-fourths) of 
outcome indicators in the consensus-based NCEO Conceptual Model of Outcomes (School 
Completion) are present across 13 national data collection programs. The bad news is that this 
potential cannot be tapped currently to analyze and generate reports about students with disabilities. 
The exclusion of significant numbers of students with disabilities from recurring national data 
collection programs and the variable or nonexistent disability-specific variables used to identify 
students in these data collection programs makes it all but impossible to use all the outcome 
information that is currently available at the national level. 

Recommendation s 

The inability to extract comprehensive national outcome information at school completion for 
students with disabilities is not due to a lack of potential indicators in our national data collection 
programs. The exclusion of significant numbers of students with disabilities from many national 
data collection programs and the variable identification of these students by type of disability in 
these programs are the greatest impediments to the production of recurring national reports on the 
school completion status of students with disabilities. Recommendations to address these two 
concerns have been presented elsewhere (McGrew, Thurlow & Spiegel, 1993; McGrew, Thurlow, 
Shriner, & Spiegel, 1992). 

However, improvements can still be made in the types of items included in national data 
collection programs. The goal of these improvements would be to increase the breadth of 
information on all students. Individuals and organizations charged with the design of national data 
collection programs are encouraged to review the following general suggestions. 

L Attention needs to be directed to the development of indicators related to accessing 
health care, advocating for themselves, accepting the consequences of their behavior, 
and satisfaction with education. The specific NCEO indicators are: 

- Percent of students who know when, where, and how to access health care (Cld) 

- Percent of students who effectively advocate for themselves (D2c) 

- Percent of students whose behavior reflects an acceptance of the consequences for 
behavior (e.g., makes restitution) (Gib) 

- Percent of students who are satisfied with level of achievement (HI a) 

- Percent of parents/guardians who are satisfied with levei of achievement (H2a) 

- Percent of community (teachers, policymakers, employers, general public) who are 
satisfied with students 1 level of achievement (H3a) 

- Percent of community (teachers, policymakers, employers, general public) satisfied, 
with students' progress toward achieving educational outcomes (G3a). 

2. The NCEO conceptual model of outcomes should be reviewed for ideas on new 
outcomes or indicators to include in ongoing or future national studies. Consideration 
needs to be given to the development of variables to better measure outcomes related to 
satisfaction, accommodation and adaptation, personal and social adjustment, and 
responsibility and independence. 

3. Individuals and organizations developing instrumentation for national data collection 
programs should review the instruments and methodology reports of the National 
Longitudinal Transition Study of Special Education Students (NLTS) for ideas on how 
to develop new indicators or strategies for measuring outcomes for all students. 
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